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Abstract In this paper, we study the reflected BSDE with one continuous 
barrier, under the monotonicity and general increasing condition on y and 
non Lipschitz condition on z. We prove the existence and uniqueness of the 
solution to these equation by approximation method. 
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1 Introduction 

Nonlinear backward stochastic differential equations (BSDE in short) were 
firstly introduced by Pardoux and Peng in 1990, [12]. They proved that 
there exists a unique solution (Y, Z) to this equation if the terminal condi- 
tion £ and coefficient / satisfy smooth square-integrability assumptions and 
f(t,uj,y,z) is Lipschitz in (y,z) uniformy in (t,u). Later many assumptions 
have been made to relax the Lipschitz condition on /. Pardoux (1999, [IT] ) 
and Briand et al. (2003, [TJ) studied the solution of a BSDE with a coefficient 
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f(t, u), y, z), which still satisfies the Lipschitz condition on z, but only mono- 
tonicity, continuity and generalized increasing on y, i.e. for some continuous 
increasing function tp : R + — > R + , real number \i > 0: 

\f(t,y,0)\ < |/(i,0,0)| + ^(M),V(t,|/)e[0 ) r]xE, a.s.; (1) 

- y, *) fit, y', *)) < Kv - y')\ v(t, *) e [o, r] x M d , y , y ' e r, a.s. 

The case when / is quadratic on z and £ is bounded was firstly studied 
by Kobylanski in [BJ. She proved an existence result when the coefficient 
is only linear growth in y, and quadratic in z. In [9], Lepeltier and San 
Martin generalized to a superlinear case in y. More recently, in [2], they and 
Briand considered the BSDE whose coefficient / satisfies only monotonicity, 
continuity and generalized increasing on y, and quadratic or linear increasing 
in z, i.e. 

{y - y'){f{t, y, z) f(t, y', z)) < fx{y - y') 2 , V(t, z) E [0, T] x R d , y, y' E E, a.s. 

< ^(|y|)+A|z| 2 , V(t,y) e [0,T] xR, a.s.; (2) 

or 

\f(t,y,z)\<g t + tp(\y\) + A\z\,V(t,y) e [0,T] xR, a.s.. (3) 

In the same paper, they studied the case f(t,y,z) = \z\ p , for p e (1,2], 
and gave some sufficient and necessary conditions on £ for the existence of 
solutions. 

El Karoui, Kapoudjian, Pardoux, Peng and Quenez introduced the notion 
of reflected BSDE (RBSDE in short) on one lower barrier in 1997, [1]: the 
solution is forced to remain above a continuous process, which is considered 
as the lower barrier. More precisely, a solution for such equation associated 
to a coefficient f(t,u,y,z), a terminal value £, a continuous barrier L, is a 
triple (Y t , Z t , K t )o<t<T of adapted processes valued on R 1+d+1 , which satisfies 
a square integrability condition, 

Y t = i + J^ f(s,Y a ,Z a )ds + K T -K t - J Z s dB s ,0<t<T, a .s., 

and Y t > L t , < t < T, a.s.. Furthermore, the process (K t ) < t < T is non 
decreasing, continuous, and the role of K t is to push upward the state rjrocess 
in a minimal way, to keep it above L. In this sense it satisfies J (Y s — 
L s )dK s = 0. They proved the existence and uniqueness of the solution when 
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f(t,u,y,z) is Lipschitz in (y,z) uniformly in (t,u). Then Matoussi (1997, 
|lUj ) consider RBSDE's where the coefficient / is continuous and at most 
linear growth in y, z. In this case, he proved the existence of maximal 
solution for the RBSDE. 

In [TJ, Kobylanski, Lepeltier, Quenez and Torres proved the existence of a 
maximal and minimal bounded solution for the RBSDE when the coefficient 
f(t,u, y, z) is super linear increasing in y and quadratic in z, i.e. there exists 
a function I strictly positive such that 

f°° dx 

\f{t,y,z)\ <l{y) + A\z\\ with / — - = +00. 

Jo l \ x ) 

In this case, £ and L are required to be bounded, and L is a continuous 
process. Recently, in [8] Lepeltier, Matoussi and Xu considered the case when 
f(t,u,y,z) satisfies (pQ) and is Lipschitz in z. They proved the existence and 
uniqueness of the solution by an approximation procedure. 

In this paper, we study the RBSDEs whose the coefficient / satisfies the 
conditions (j2J) or ([3]), when the lower barrier L is uniformly bounded. We 
prove the existence of a solution, following the methods in [2], and we give a 
necessary and sufficient condition for the case when f(t,u,y,z) = \z\ 2 , and 
its explicit solution. 

The paper is organized as follows: in Section 2, we present the basic 
assumptions and the definition of the RBSDE; then in Section 3, we prove 
the existence of a solution when f(t,u>,y,z) satisfies the conditions (j2j), £ 
and L are bounded; in the following section, we consider the case when 
f(t,u,y,z) = \z\ 2 , and £ is not necessarily bounded. In this section, we give 
a necessary and sufficient condition on the terminal condition £ for p = 2 
and its explicit solution. Finally, in section 5, we study the RBSDE with the 
condition ([3]), and prove the existence of a solution. At last, in Appendix, we 
generalize the comparison theorem in [7] , and get some comparison theorems, 
which help us to pass to the limit in the approximations. 



2 Notations 

Let (ft, J 7 , P) be a complete probability space, and (B t )o<t<T = (B], B%, • ■ ■ , Bf)' 0<t<T 
be a d- dimensional Brownian motion defined on a finite interval [0, T], < 
T < +00. Denote by {J-" t ; < t < T} the standard filtration generated by 
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the Brownian motion B, i.e. Tt is the completion of 



T t = a{B s ; < s < t}, 

with respect to (J 7 , P) . We denote by V the cr-algebra of predictable sets on 
[0,T] x tt. 

We will need the following spaces: 

L 2 (jF 4 ) = {rj : .^-measurable random real-valued variable, s.t. _E(|?7| 2 ) < +00}, 
H 2 (0,T) = {{tpt)o<t<T '■ predictable process valued in IR n , s.t. E f Q \ip(t)\ 2 dt < +00}, 
S 2 (0,T) = {(ip t )o<t<T '■ progressively measurable, continuous, real-valued process, 

s.t. £(sup < t < T |^0)| 2 ) < +00}, 
A 2 (0,T) = {(K t ) < t < T : adapted continuous increasing process, 

s.t. K(0) = 0, E(K(T) 2 ) < +00}. 

Now we introduce the definition of the solution of reflected backward 
stochastic differential equation with a terminal condition £, a coefficient / 
and a continuous reflecting lower barrier L(in short RBSDE(£, /, L)), which 
is the same as in El Karoui et al.(1997, [1]). 

Definition 2.1 We say that the triple (Y t , Z t , K t )o<t<T of progressively mea- 
surable processes is a solution of RBSDE(^, f, L), if the followings hold: 

(i) (Y t )o<t< T e S 2 (0,T) ; (Z t ) <i<r e H 2 (0,T) ; and (K t ) < t < T e A 2 (0,T). 

(ii) Y t = i + f(s, Y S1 Z s )ds + K T -K t - jff Z s dB s , < t < T a.s. 
(m) Y t > L t , < t < T. 

(w) J*(Y s -L s )dK s = 0, a.s. 



3 The general case of / quadratic increasing 

In this section, we work under the following assumptions: 

Assumption 1. £ is an jF T -adapted and bounded random variable; 
Assumption 2. a coefficient / : Q x [0, T] such that 

for some continuous increasing function tp : R + — > M + , real numbers \i and 

A > and V(t, y, y'z) 6[0,T]xKxlx M d , 

(i) /(■, y, z) is progressively measurable; 

(ii) \f(t,y,z)\ <^\y\)+A\z\ 2 - 

(iii) (y - y'){f{t, y, z) - f(t, y', z)) < M (y - y') 2 ; 

(iv) y — > f(t,y, z) is continuous, a.s. 
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Assumption 3. a barrier (L t ) <t<T, is a bounded continuous progres- 
sively measurable real-valued process, b := sup 0<t<T \L t \ < +00, L T < £, 
a.s. 

Then we present our main result in this section. 

Theorem 3.1 Under the Assumptions 1, 2 and 3, RBSDE(£, f, L) ad- 
mits a maximal bounded solution. 

Proof. First, notice that (Y, Z, K) is the solution of RBSDE(£, /, L) if and 
only if (Y\ Z\ K b ) is the solution of the RBSDE(f 6 , f b , L b ), where 

(Y b ,Z b ,K b ) = (Y-b, Z,K), 

and 

(C\ f% y, z),L b ) = (£-&, f(s, y + b,z),L-b). 

Notice that (£ b , f b , L b ) satisfies Assumption 1, 2 and -26 < L b < 0. So in 
the following, we assume that the barrier L is a negative bounded process. 
For C > 0, set g : R — > R be a continuous function, such that < 

g c (y) < l, Vy e R, and 

<7 C (y) = 1, if |y| < C, (4) 
g°{y) = 0, if \y\ > 2C. 

Denote / c (t, y, z) = g c (y)f(t, y, z); then 

\f c (t,y,z)\ < g c {y){ip{\y\)+A\z\ 2 ) 

< l[-2c,2c](y)M\y\) + A\z\ 2 ) 

< (p(2C) + A\z\ 2 . 

From the theorem 1 in [7], there exists a maximal solution (Y c , Z , K' 7 ) to 
the RBSDE(£,/ C ,L) 

F f c = £ + £ g c (Y s c )f(s,Y s c ,zC)ds- £ ZfdB s + K% - K°, (5) 

Y t c > Lt, [ {Y t ° - L t )dK? = 0, a.e.. 
Jo 
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We choose n > 2 even, and a G M; applying Ito's formula to e at (Y t c ) n , we 
have 

e at (Y t c r = e aT C + nj T e as (Y s c r- 1 g c (Y s c )f(s, Y s ° \ Z c s )ds -n^ e as {Y^ Z c s 'dB s 
- n(n ~ 1} ^ e as (Y s c ) n ~ 2 \zC\ 2 ds + n£ T e as (lf)"-Wf - a jT e as (F s c ) 

From Assumption 2 and the fact that n is even, we have 

yf(s,y,z) < yf(s,0,z) + [iy 2 , 
y n - l f{s,y,z) < y n - 1 f(s,0,z) + w n . 

With < g c '(y) < 1, we get 

9 C (v)lT l f{8,V,z) < g c (y)\y\ n - 1 f(s,0,z)+w n 

< g c (y)\y\ n - 1 (m + A\z\ 2 )+ f iy n 

< (i + — |y|>(0) + A ^(y) + ah," 

n n 

< (l + y n )^(0) + 2CA\z\ 2 y n - 2 + fiy n . 
Substitute it into (jSJ), then 



e at(Y, c ) n < e al C + -^-^(e al - e at ) + (mp(Q) + n/z - a) / e as (y s c ) n ds 

a ' J t 

+ (2nCA- n{n ~ l) ) j\ as (Y s c ) n - 2 \Z^\ 2 ds + n j\ as (L^dR? 
-n [ e as (yC) n - x Z c s dB s . 



Notice that since K c is an increasing process, n is even and L < 0, we get 
immediately 

j e as {L s ) n - l dK° < 0. 
If we choose n and a satisfying 

n - 1 > 4 CM, a = n(y>(0) + /x), 
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e at (Y t c ) n < e aT C + ^l( e aT - e at ) - n F e as {Yff- x Z c s dB s 



then 



It follows that 

e at {Y C )n < E[e aT {e + ^m m] < e^dleUSo + 1), 

at last we get 

{Y c )n < e-CT-Odlfll" + i) < ( e ^ V 1)(||£||» + 1)- 
Since a = n((p(0) + fi), it follows that 

\Y t °\ < (e^°)W V 1)(U\L + 1)- < (e^°)+^ T V l)(||eiL + 1). 

If C is chosen to satisfy C > ( e ^+^ T V 1)(||C|L + 1), then we have 
\Y t c \ < C, which implies g c {Y t c ) = 1, for < t < T. So, (Y c , Z c , K c ) is 
the solution of the RBSDE(£, /, L). □ 

2 

4 The case f(t,y,z) = \z\ 

In this section we consider the case f(t,y,z) = \z\ 2 , which corresponds to 
the RBSDE 

Y t = e+ / |^| 2 ^ + K T -K t - /" Z s oLB s , (7) 

T 



^ > L t , / (r t - L t )dtf t = 0. 
Jo 



Then we have 

Theorem 4.1 Under the assumption E(sup 0<t<T e 2Lt ) < +oo, the RBSDE(£, f, L) 
(0) admits a solution if and only if E(e 2 ^) < +oo. 

Proof. For the necessary part, let (Y, Z, K) be a solution of the RBSDE 
((7j). By Ito's formula, we get 

e 2Y t = e 2 5 + 2 y T e 2n d ^ _ 2 ^ e Ys Z4Bs (g) 
t /•* 



3 2Yo^ / e 2Y sZsdjBs _ 2 / e 2Y s 
'0 Jo 
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Let for all n, r n = M{t : Y t > n} A T, then M tATn = 2 J ' ATn e 2Ys Z s dB s is a 
martingale, and we have 

E[e 2Y ^] = E[e 2Y ° - 2 f e 2Y *dK s \ < E[e 2Y °], 

Jo 

m view of 2 J*e 2Y °dK s > 0. Finally, since T n /* T, when n — ► oo: 

^[hm^e 2 ^] = E[e 2 ^] < E[e 2Y °} < oo, 

follows from Fatou's Lemma. 

Now we suppose E(e 2 ^) < +oo, set L t = L t l{t<T} + £l{t=T} and 

N t = S t (e 21 ) = ess sup E{e 2L ^\T t \, 

where S t (fj) denotes the Snell envelope of r\ (See El Karoui [3]), T ti r is the 
set of all stopping times valued in [t, T\. Since 

E[ sup e 2Lt ] < E[ sup e 2Lt + e 2? ] < +oo, 

0<t<T 0<t<T 

using the results of Snell envelope, we know that N is a supermartingale, so 
it admits the following decomposition: for an increasing integrable process 
K, 

N t = N + ! Z s dB s -K t . 



Applying Ito's formula to logA^, we get 

l -\ogN t = -\ogN Q + - f^dB s -- ['{Elfds-- f —dK s . 
Set Y t = -log A 7 "^ Z t = K t = ~ f* j^dK s , then the triple satisfies 

= £ + y Z 2 s ds + K T -K t - Z s dB s . (9) 

Thanks to the results on the Snell envelope, we know that A t > e 2L * and 
J T (A t - e 2Lt )d~K t = 0. The first implies 



Yt > L t > L 
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Obviously, N t > 0, < t < T, so K is increasing. Consider the stopping 
time D t := inf{£ < u < T; Y u = L u } A T, then it satisfies D t = inf{£ < u < 
T; 7V M = e 2Lu } A T. By the continuity of K, we get K Dt -K t = 0, which 
implies K]j t — K t = 0. It follows that 



/ V* - L t )dK t = 0. 
Jo 



Now the rest is to prove Y t E S 2 (0,T), Z t G H 2 (0,T), and e A 2 (0,T). 
With Jensen's inequality 

Y t = hog N t = l\og[ess sup E[^\T t \] 

1 1 reT tyT 

> - log[exp(ess sup £ , [2L T |^ r t ])] 
= ess sup E[L T \F t \ > E[£\F t ] > U t , 

reTt,T 

where U t = — E[^~\J^. For all a > 0, define 

r a = mf{t; \N t \ > a, (j^) 2ds > <*> / W dBf 

J Q S J £ 

From (JSP, we get for < t < T 



> a}. 



Then 



< / Z 2 s ds = Y -Y t + I Z s dB s - K t 



lo 



< Yo-U t 



o 



( f " Z 2 s dsf < 3(F ) 2 + 3(/7 T J 2 + 3( / ° Z s dB s f. 
Jo Jo 



Taking the expectation, using the Jensen's inequality and 3x < y + |, we 
obtain 

9 

. : Hi I 7r,ls)r - 

1 

2 



EiTzldsf < l(\ogN ) 2 + 3E(r) 2 + ^E(l/ Z») 2 + 



< l(logN f + 3E(C) 2 + ^E(l ~Z 2 s ds)^-, 
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so 

E(£ Z 2 ds) 2 < \{\ogN,) 2 + 6E(C) 2 + 9 < C. 

Since r a /* T when a — > +00, we get to the limit, and with the Schwartz 
inequality 

E [ Z 2 s ds < (E( [ Z 2 ds) 2 )^ < C. 
Jo Jo 

So Z e H 2 (0, T). From ©, we get for < t < T 

< K t — Y — Y t + [ Z s dB s - [ Z 2 ds 

Jo Jo 

< Y -Y t + [ Z s dB s . 
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Notice that K is increasing, so it's sufficient to prove E[K%] < +00. Squaring 
the inequality on both sides and taking expectation, we obtain 

E[(K T ) 2 } < 3Y 2 + 3E[£ 2 ] +3E [ Z 2 ds < C. 

Jo 

We consider now Y; again from ([9]), 

Y t = Y -K t + [ Z s dB s - I Z 2 ds, 
Jo Jo 

SO 2 2 

(Y t ) 2 < 4 (Y ) 2 + 4 (K t f + 4 Qf Z s dB s j + 4 ^ Z 2 ds^j . 
Then by the Bukholder-Davis-Gundy inequality, we get 

E[ sup (Y t ) 2 ] < 4 (Y ) 2 + AE[K 2 } + AE[ sup ( [ Z s dB s ) ] + 4E ( f Z 2 ds 

0<t<T 0<t<T \Jo J \Jo 

< 4 (y ) 2 + 4E[K 2 } + CE (j Z 2 dB s ^ + AE (J Z 2 ds^J < C 
i.e. Y G S 2 (0,T). □ 
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5 The case when / is linear increasing in z 

In this section, we assume that the coefficient / satisfies 

Assumption 6. (i) /(•, y, z) is progressively measurable, and E J Q f 2 (t, 0, 0)dt < 
+oo; 

(ii) for // G R, V(t, z) G [0, T] x R d and y, y' G R, 

(y - y')(f(t, y, z) - f(t, y', z)) < »(y - y'f- 

(iii) there exists a nonegative, continuous, increasing function (p : K + — > 
R+ with y?(0) = 0, s.t. V(t, y, z) 6 [0,T] xlx R d , 

\f(t,y,z)\ < \gt\ + <p(\y\)+P\z\, 

where g t G H 2 (0,T); 

(iv) for £ G [0, T], (y, 2) — * f(t, y, z) is continuous. 

If <p(x) = \x\, then / is linear increasing in y and z. Matoussi proved in 
[TO] that when f G L 2 (J" T ) and L G S 2 (0,T), there exists a triple (Y,Z,K) 
which is solution of the RBSDE(£, /, L). 

Our result of this section is the following: 

Theorem 5.1 Suppose that £ G L 2 (jF T ), / and L satisfy Assumption 6 

and 3, respectively, then the RBSDE(£, f, L) has a minimal solution (Y, Z, K) G 
S 2 (0,T) x H 2 (0,T) x A 2 (0,T) ; which satisfies 



Y t = { + J T f{s,Y s ,Z s )ds + K T -K t - J T Z 



s dB s , 



Y t > Lt, and f*(Y 8 - L s )dK s = 0. 

First we note that the triple (Y, Z, K) solves the RBSDE(£, /, L), if and 
only if the triple 

(Y t ,Z t ,Kt) ■= (e xt Y t ,e xt Z t , [ e Xs dK s ) (10) 

Jo 

solves the RBSDE(£, /, L), where 

(£, J(t, y, z),L t ) = (£e AT , e xt f(t, e~ xt y, e^z) - Xy, e xt L t ). 
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If we choose A = a, then the coefficient / satisfies the same assumptions 
as in Assumption 6, with (ii) replaced by 
(if) (y-y')(f(t,y,z)-f(t,y',z))<0. 

Since we are in the 1-dimensional case, (ii') means that / is decreasing on 
y. From another part £ still belongs to L 2 (JF T ) and the barrier L still satisfy 
the assumptions Assumption 3. So in the following, we shall work under 
Assumption 6' with (ii) replaced by (ii'). 

Before proving this theorem, we consider an estimate result and a mono- 
tonic stability theorem for RBSDEs. 

Lemma 5.1 We consider RBSDE(^, g, L), with £ e L 2 (jF T ) ; g and L satisfy 
Assumption 6' and 3. Moreover g(t,y, z) is Lipschitz in z. Then vie have 
the following estimation 



E[ sup \y t \ 2 + / 

0<t<T Jo 



\z s \ ds + Ikx] 2 ] 



< C p E[\i\ 2 + [ T g 2 s ds + ip 2 (b) + ^(2T) + l] 
Jo 

where (y t , z t , k t ) Q < t < T is the solution of RBSDE(^, g, L) . Cp is a constant 
only depends on (3, T and b. 

Remark 5.1 The constant Cp does not depend on Lipschitz coefficient of g 
on z. 

Proof. Since g is Lipschitz in z, by the theorem 2 in [8], the RBSDE(£, g, L) 
admits the unique solution (y t , z t , k t ) < t <T- Apply Ito's formula to \yt\ 2 , in 
view of yg(t, y, z) < g(t, 0, 0) \y\ + (3 \y\ \z\ and sup < t < T \L t \ < b, we get 

E[\y t \ 2 + \zfds] = E[\£\ 2 + 2 y s g(s,y s ,z s )ds + 2 L s dk s ] 
Jt Jt Jt 

< E[\Z\ 2 + 2j^ y s g s ds + 2(3j y s z s ds + 2b{k T - k t ) 



It follows that 



E[\yt\ 2 



< 



1 / 

2 J t 

E[\Z\ 2 + J t 9 2 s ds+ (l + 2(3 2 ) J \y s \ 2 ds + 2b(k T -k t )]. 
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By Gronwall's inequality, we know there exists a constant c\ depending on 
(3 and T, such that for t E [0, T], 

E[\y t \ 2 } <c 1 E[\^\ 2 + f g 2 s ds + b(k T -k t )}. (11) 
Jo 



It follows that 



E[f \z s \ 2 ds]< 2(1 + (1 + 2P 2 )T) C1 E[\^\ 2 + [ glds + b{k T -k t )}. (12) 
Jt Jo 

Now we estimate the increasing process k by approximation. Take z as 
a known process, without losing of generality, we write g(t,y) for g(t,y,z t ), 
here g(t, 0) = g(t,0,z t ) is a process in H 2 (0,T) in view of linear increasing 
property of p on z. 

For m, p G N, set £ m > p = (f V (-p)) A m, # m ' p (t, u) = g(t, u) - g t + (g t V 
(-p)) A m. We consider RBSDE^, L), 



Vt 



£ m * + £ g m ' p {s,y™> p )ds + k™' p -k™' p - £ z™' p dB s , (13) 

?/r p > l*. A^r - L t )dkr = o. 



It is easy to check that (y m ' p , z m ' p , k m ' p ) is the solution of RBSDE(£ m ' p , 9 m ' p , L), 
if and only if (y m >P,2 m >P, k m ' p ) is the solution of RBSDE(£ m ' P , g" 1 ^, L), where 

(^,^,fcr p ) = (yT' p +m(t-2(Tvi)),zr' p ,krn, 

and 

£ m - p = £™>P + m T-2m(rvl), 
y) = r p (i,!/-m(i-2(TVl)))-m, 
L t = L ( + m(t-2(TVl)). 

Without losing of generality, we set T > 1. Since £ m ' p and g™ ,p < m, we 
have ? m ' P and g?> p < 0. By @3}, 
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taking square and expectation on the both sides, we get 

E[(k^ p -kT' p ) 2 } <4El(yT' P ) 2 +(Ty+( f r> P (s,y?' p )ds) 2 + f\^ p \ 2 ds]. 

(14) 

In order to estimate the first and the last form on the left side, we apply 
Ito's formula to |2/J™' P | 2 , and get the following with Gronwall inequality, 

E[\y?> p \ 2 + J \z?«\ 2 ds) (15) 

< c 2 £[|? m ' P | 2 + J T (g?< p ) 2 ds + f L s dk™% 

where c 2 is a constant only depends on T. For the third term, let us recall a 
comparison result of y^' p in step 2 of the proof of theorem 2 in [8] , 

Vt <Vt < Vt , 
where y^' p is the solution of BSDE(£ P ,g m,p ), i.e. 

+ f gr*{s,V?*)d8 - [ T z?' p dB s , (16) 
Jt Jt 

and 

y™' p = ess sup E[(L T ) + l {T<T} + (f ' ) + l {r=T} |^], 

reT t!T 

where is the set of stoppng times valued in [t,T]. Moreover, we have 

sup < s <tKT' p = sup < s < T L s . 

Since 'g m,p is decreasing in y, we get 

g m ' p (s,y?' p ) < r ,p (5,r ,p ) < r' p (s,yT' p )- 

So to estimate E^f^ g m > p (s, y^ ,p )ds) 2 \, it is sufficante to get the estimations 
of E[(ff ^(s^^ds) 2 ] and E[(f?g m * (s, y^' p )ds) 2 }. First we know that 

E[{f 9 m *(s,V?*)d8)*\ < 3E[\T' P \\ \yT' P \ 2 + \^T' P \ 2 ds] (17) 

< c 3 E[\T' P \\ j\g?' P ) 2 ds], 
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in view of estimate result of BSDE (TiT)|) . Here c 3 is a constant only depends 
on T. Then with the presentation of y] a,p , we have 



E[( f g m > P (s,y?' p )ds) 2 ] < E[2T C ®?*) 2 ds + 27> 2 ( snp (L t )+)] (18) 

Jt JO 0<t<T 



From ([MD, with (PT5D. (flTD and (fl8l. we have 



< c 4 i?[e +/ (gT' p ) 2 ds+ L s dk^ p + ^ 2 ( sup (Lt) 



r 



0<t<T 



< c 4 £[2 |C iP | 2 + ij\gT' p ) 2 ds + 2c 4 b 2 + y? 2 (6)] 
+Ac A {m 2 T 2 + ^ 2 (2mT)) + - Af' p ) 2 ], 

where c 4 = c 2 V c 3 V (2T), which only depends on T. It follows that 
E[{k^ p - k™' p ) 2 } 

< c 5 E[\C' p \ 2 + J T (gT' p ) 2 ds + b 2 + v 2 (b)] + c 5 (m 2 T 2 + ^ 2 (2mT)) 

r T 

< c 5 E[\£\ 2 + J g 2 ds + b 2 + ip 2 (b)] + c 5 (m 2 T 2 + ip 2 (2mT)), 

where c 5 = 4c| V 8c 4 . 

Now we consider the RBSDE(£ P , g p , L), where f p = £ V (-p), # p (t,w) = 
u) — g t + (?i V (— p) . Thanks to the convergence result in [8] , we know that 

( y m,p^ z m, Pjk m,p} (jjP ^ fcP) in s 2 (0,T) x H 2 (0, T) x A 2 (0,T), 

where (y p ,z p ,k p ) is the solution of RBSDE(£ P , g p , L). Moreover, we have 
dk P < dkl ,p } by comparison theorem. So 

E[(k p T -k P ) 2 ] < E[{k l /-k l *y] 

< c 5 E[\ti\ 2 + [ glds + b 2 + ^ 2 {b)]+c,{T 2 + V 2 {2T)). 



Then let p — > oo, thanks to the convergence result in [8], we know 
(^,^,fcP) - (y,z,k) in S 2 (0,T) x H 2 (0,T) x A 2 (0,T). 
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In view Assumption 6-(iii), it follows that 

E[{k T -k t f] < c,E[\i\ 2 + j\g{s^z s )fds + b 2 + V 2 {b)} 

rp rp 

< c 5 E[\£\ 2 + 2 J g 2 ds + 2(3 2 J \z s \ 2 ds + b 2 + ^ 2 (b)} 
+c 5 (T 2 + ^ 2 (2T)). 

With ([12]), setting c 6 = c 5 V (4/9 2 (l + (1 + 20 2 )T)c 1 + 2) V c 5 (b 2 + T 2 ), we get 
E[{k T -k t ) 2 } < C6 E[\C\ 2 + j\ 2 ds + b(k T -k t )+ip 2 (b) + ip 2 (2T) + l] 

rp 

< c 6 E[\i\ 2 + ^ g 2 ds + 2c 6 b 2 + V 2 (b) + <p 2 (2T) + l] 
+ 1 -E[(k T - k t ) 2 ] 

It follows that 

E[(k T - k t ) 2 ] < 2c 6 E[\^\ 2 + 2 J g 2 ds + cp 2 {b) + cp 2 (2T) + 2c 6 b 2 + I). 
Consequantly, by ffTTl) and ffT2l . we obtain 



\k T \ 2 ] 



sup E[\y t \ 2 ] +E[ \z s \ ds + 

0<t<T Jo 

< C p E[\i\ 2 + [ T g 2 ds + ^ 2 (2T) + if 2 (b) + l], 
Jo 

where Cp is a constant only depends on (3, T and b. The final result follows 
from BDG inequality. □ 

The proof of this theorem is step 1 and step 2 of the proof of theorem 4 
in [7], with comparison theorem. So we omit it. 

With these preparations, we begin our main proof. 

Proof of theorem 15.11 The proof consists 4 step. 

Step 1. Approximation. For n>(3,we introduce the following functions 

fn(t, V, z) = inf {f(t, y,q) + n\z- q\}, 

q&Q d 
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then we have 

1. for all (t,z), y — > f n (t,y,z) is non-increasing; 

2. for all (t, y), z — ► f n (t,y,z) is n-Lipschitz; 

3. for all (t,y,z), \f n (t,y,z)\ < \g t \ + cp(\y\) + P \z\ . 

Thanks to the results of [8], we know that for each n > [3, there exits a 
unique triple (Y n , Z n , K n ) satisfies the followings 

Y? = £ + f f n (s,Y?,Z«)ds + K%- K? - J T Z^dB s , 

Y t n > L t , f{Y t n - L t )dK? = 0. 
Jo 

Step 2. Estimates results. Let a > 0, be a real number to be chosen 
later. We set 

jjn = e at Y n^ yn = %n ^ dJ n = ^t^n 

Then we know that (U n ,V n ,J n ) is the solution of the RBSDE associated 
with ((,F n ,L a ), where 

C = e QT e, F n (t, u, v) = e at f n (t, e" Q ^, e~ at v) - au, L a t = e at L t . 

It is easy to check 

\F n (t, u,v)\< e at \g t \ + e at (p(\u\) + a \u\ + (3 \v\ , 

setting i/j(u) = e aT <p(\u\) + a\u\, with ip(u) = 0, we get that F n verifies 
Assumption 6'-(iii). Moreover 

uF n {t,u,v) = e at uf n (t, e~ at u, e~ at v) - au 2 
< ue at g t + (3 \u\ \v\ — au 2 . 

And sup 0<i<r Lf < e aT sup 0<t<T L t < e aT b. Now we apply Ito formula to 
\U n \ 2 on [0,T], and get 

\ U t\ 2 + [ \ V s n \ 2 ds 

= ici 2 + 2 f u:F n ( S , u:, v:)ds + 2 f u?<u? - 2 f u:v s n dB s 
jt Jt Jt 

< |C| 2 + f ' e 2as g 2 s ds + (l + 2(] 2 -a) j" Vjf ds + \ f \V s n \ds 

1 r T 

+9e 2aT b 2 + -(J£ - Jf ) 2 - 2 / U^V s n dB s , 
v Jt 
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where 6 is a constant to be decided later. By taking conditional expectation, 
we get 

\U?\ 2 + ±E[J T \V s n \ 2 ds\F t ) < E{\(\ 2 + j\ 2as g 2 s ds + ee 2aT b 2 \F t } (19) 

+(1 + 2(3 2 - a)E[J T \U:\ 2 ds\F t ] + l -E[{Jl - J?f\F t ]. 

Since 

J2-J? = U?-<;- J F n (s,U:,V s n )ds- J V s n dB s , 

we have 

E[(JZ-J?)'\F t ] < 4 \U-\ 2 +AE[\C\ 2 +{[ F n (s, U?, V?)daf+ £ \V?\ 2 ds\T t }. 

Using the same approximation as in Lemma 15.11 except considering condi- 
tional expectation i?[-|.Ft] instead of expectation, we deduce 

E[(J$ - J?) 2 \F t ) < c^E[\C\ 2 + jT e 2as g 2 ds + ^ 2 (e aT b) + ^ 2 {2T) + l|.F t ], 

where c@ is a constant which only depends on (3, T, b and a. Substitute it 
into (fl9~j) . set a = 1 + 2(3 2 , 6 = c@, then we get, 

\U?\ 2 < 2E[\(\ 2 + ^ F 2 (s,0,0)ds\F t ]+e aT ( V (e aT b) + v(2T)) 
+a{e aT b + 2T) + 1 + c p e 2aT b 2 . 
Recall the definition of U n , we get 

\Y t n \ 2 < e- 2a \2E{e 2aT \£\ 2 + J* e 2as g 2 ds\F t ] + e aT (ip(e aT b)+tp(2T)) 
+a(e aT b + 2T) + l + cpe 2aT b 2 ). 

If we set M t = (e 2aT 2E[\£\ 2 + jf g 2 s ds\F t } + e aT (ip(e aT b) + ip(2T)) + Cfie 2aT b 2 + 
a{e aT b + 2T) + 1)1, then 

|Y t "| < M t ,V* 6 [0,T]. (20) 
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Step 3. Localisation. 

First, we know that the sequence (f n ) n >p is non-decreasing in n, then 
from comparison theorem in [8], we get 

Y n < Y t n+1 ,Vt G [0,T],Vn > 0. 

Define Y t = swp n > p Y t n . 

We now consider the localisation procedure. For m G N, m > b, let r m 
be the following stopping time 

r m = mf{t G [0, T] : M t + g t > m} A T, 

and we introduce the stopped process Y™' 171 = , together with Z^ ,m = 
z t l {t<r m } and Kt' m = K^ hTm . Then (Y t n ' m , Z"' m , K^ m ) < t < T solved the 
following RBSDE 

yn,m = + f l^^f^ Y ?>™ , Z^)d S + - K? m - F Z? m dB„ 

Jt Jt 

Yr > Lt, F(Y t n > m - L t )dK r t 
Jo 



where £ n,m = K n,m = K n . 

Since {Y n ' m ) n >p is non-decreasing in n, with (I2U|) . we get sup n>/3 sup te r 0jT i |^ n,m | < 
m. Set p m (y) = max ^ m} , then it is easy to check that (Y n > m ~Z n ' m , K rl > m ) 
verifies 

yn,m = C , m + j\ {s ^ }fn ^ pm{ yn, mlz n, m)ds ^ 

Y t n ' m > Lt, (Y t n ' m - L t )dK^ m = 0. 



Moreover, we have 

\~L{s<T m }fn(s,p m (y),z)\ <m + ip(m)+(3\z\ , 
and |£ n,m | < 77i. From Dini's theorem, we know that l{ s <-r m } 

fn{s,p m {y),z) 

converge increasingly tol{ s < Tm i/(s, p m (y), z) uniformly on compact set of 
R x K , because / n are continuous and / n converge increasingly to /. And 
£"' m converge increasingly to £ m a.s., where £ m = sup n>/3 £"' m . 
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As in [Til], we can prove that Y n,m converges increasingly to Y m in 
S 2 (0,T), and Z n ' m -> Z m in H^(0,T), K n ' m \ if m uniformly on [0,T]. 
Moreover, (T m ,Z m ,ir m ) solves the following RBSDE 

*T = C + fhs<T m }f(s,P m (Y s n,Z?)ds + K™-K?-J T Z™dB s , 

Y t m > L t , [ T (Y t m - L t )dK? = 0, 
Jo 

where £ m = sup n>(3 Y^ 71 . Notice that \Y t m \ < m, so we have 

Y t m = T + f ks<r m }f(s, Y s m , ZT)ds + K™- K™ - ^ Z[?dB s . 

From the definition of {r m }, it is easy to check that r m < r m+ i, with the 
definition of Y m , Z m , K m and Y, we get 

1 tAr m — *tAr m ~ Y t 1 L {t<T m } ~ ' A iAr m ~ A i ■ 

We define 

Z t := Zll {t < Tl} + 22 Z T l (r m -ur m ]{t), K tATm := K™ . 



m>2 



Processes (Y m ) are continuous, and F-a.s. r m = T, for m large enough, 
so Y is continuous on [0,T]. It follows that K is also continuous on [0, T]. 
Furthermore, we have for m G N, 

/"I'm PTm 

YtAr m = Y Tm + / f(s, Y s , Z s )ds + K Tm - K tATm - / Z s dB s . (21) 

Jtl\T m Jtl\T m 

Finally, we have 

P( I \Z S \ 2 ds = oo) = P{ I \Z S \ 2 ds = oo, r m = T) + P{ / \Z S \ 2 ds = oo, 
Jo Jo Jo 

f T 

< P{ \Z s \ 2 ds = oo) + P(r m <T), 
Jo 

in the same way, 

P(\K T \ 2 = 00) < P(\K T \ 2 = 00) + P(r m < T). 
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Since r m /* T, P-a.s., we know that J Q \Z S \ ds < oo and \Kt\ < oo, P-a.s. 
Let m — > oo in (12 II) . we get (Y, Z, X) verifies the equation. 

Step 4. We want to prove that the triple (Y, Z, iT) is a solution of 
RBSDE(£, /, L). 

First, we consider the integrability of (Y,Z,K). By fl20|) . we know for 
< t < T, 

\Y t \ < M t . (22) 

It follows immediately that 

E[ sup \Y t \ 2 ] < CpE^ 2 + / g 2 s ds + cp 2 (b) + cp 2 (2T) + 1]. 

0<t<T Jo 

where Cp is a constant only depends on f3, T and 6. For K, notice that 
K n, m ^ R m^ then for each m G o < t < T, we know < K™ < Kl' m . 

Obviously, the coefficient l{s<r m }fn(s, p m (y), z) satisfies assumption 6', and 
Lipschitz in z, by Lemma [5.11 

E[{K 1 T m ) 2 ]<C p E[\e> m \ 2 + [ T 9 2 ds + ^ 2 (b)+ V 2 (2T) + l], 

Jo 

where £ 1,m = Y* m . With flU]), we have 

E[(K^ m f] < 2C P E{\£\ 2 + f 'fids + ip 2 {b) + <p 2 (2T) + 1], 

Jo 

which follows that for each m G N, 

£[(i^) 2 ] < 2C p E[\i\ 2 + / T ^ S + <p 2 {b) + ^ 2 (2T) + 1], 

Jo 

and so does for K, i.e. we get E[(K T ) 2 ] < oo. 

In order to estimate Z, we apply Ito's formula to \Yt\ on the interval 
[0,T], then 

1 r T 

\Y f + -E / \Z s \ 2 ds 
* Jo 

< E\t\ 2 + E g 2 ds + (l + 2(3 2 )E \ Y s \ 2 ds + E[ sup | Y t \ 2 } + E[(K T ) 2 }. 

JO Jo 0<t<T 
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Thanks to the estimates for Y and K, there exists a constant C only depends 
on (3, T and b, such that 

E [ \Z S \ 2 ds < CE[\i\ 2 + I g 2 s ds + v? 2 (6) + V 2 {2T) + 1]. 
Jo Jo 

The last is to check the integral condition. Recall that (Y t m — L^dK™ = 0, 
then we have 

/ (Y t - L t )dK t = 0, a.s. 
Jo 

Since P-a.s. r m = T, for m large enough, so 

(Y t - L t )dK t = 0, a.s. 



o 



i.e. (Y,Z,K) is a solution of RBSDE(^,/,L) in S 2 (0, T) xHj(0, T) x A 2 (0, T). 
□ 



6 Appendix: Comparison theorems 

We first generalize the comparison theorem of RBSDE with superlinear quadratic 
coefficient, (in view to proposition 3.2 in [7]), to compare the increasing pro- 
cesses. Assume that Assumption 1 and 3 hold, and that the coefficient / 
satisfies: 

Assumption 7. For all (t,u), f(t,u,-,-) is continuous and there exists 
a function I strictly positive such that 



dx 



f(t, y, z) <%) + A \z\ 2 , with / -p- = +oo. 

Jo 

Proposition 6.1 Suppose that are Tx-adapted and bounded, f l (s,y,z), 
i = 1,2 satisfy the condition Assumption 7 and L satisfies Assumption 
3. The two triples (y 1 , Z 1 , fC 1 ), (Y 2 ,Z 2 ,K 2 ) are respectively solutions of 
the RBSDEfc 1 , f 1 , L) and RBSDE(^ 2 , / 2 , L). If we have V(t, y, z) G [0, T] x 

R x R d , 

e <ej\t,y,z) <f 2 (t,y,z), 
then Y t l < Y 2 , K\ > K 2 and dK\ > dK 2 , for t E [0, T] . 
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Proof. From the demonstration of theorem 1 in [7j, we know that for i = 1, 2, 
{Y\ Z\ IC) is the solution of RBSDE(f , f\ L) if and only if {9\ J\ A*) is the 
solution of RBSDE(r/\ F\L) where 

(9 i ,J i ,A i )^(exp(2AY i ),2AZ i 9 i ,2 [ A exp(2 AY s )dKl) (23) 

Jo 

and 

rf = exp(2^'),r, = exp(2 J 4L ( ) 



2 A ' 2 Ax' 4Ax 2J 

Then we use the approximation to construct a solution. For p G N, we 
consider the RBSDE(rf , F* L t ), where 

F;(s,x, A) = <?(p(0))(l - «p(A)) + *; p (A)F*( S ,p(£), A). 

Here g>(x) = 2Axi(-^), k p (A) and p(x) are smooth functions such that 
ftp(A) = 1 if |A| < p, K P (X) = if |A| > p+ 1, and p(x) = x if z G [r, 
p(x) = I if x G (0, |), p(x) = R if x G (2J2, +oo), where r and F are 
two constants. Since F % v are bounded and continuous function of (9, A), the 
RBSDE(?/, F£,~L t ) admits a bounded maximal solution (0** J** A**), with 
m < < Vo- Here m and Vo are constants given in Theorem 2 in [7j. 
We know that F % v \ F\ as p — > oo, where F* = F(s, p(0), A). Thanks to 

the proof of theorem 1 in [7], it follows that 9 l t ' p j 6 t , J l t ' p j J^, < t < T, and 
A** -> A i in H^(0,T) and (?, J*, A*) is a solution of the RBSDE(t/*, F\ L t ). 
In addition, m < 6 t < Vq. So if we choose < r < m and Vq < R, 
then F* = F\ It follows that {9 ,J\ A*) satisfies the RBSDE^, F*, I), i.e. 

($,J\A i ) = {9\J\A i ). 

Since f l {t,y,z) < f 2 (t,y,z), for (t,x,A) G [0,T] xl+xK 11 , we have 
i^foxjA) < F 2 (t,x,A). Then for p G N, Fp(s, x, A) < F 2 (s,x,A). Notice 
that Fp is bounded and continuous in (6, A) and 9 l t ' p > 0, by Lemma 2.1 in 
0, it follows that 9\ p < 9 2 t ' p , J]' p > J 2 t ' p , dj]' p > dJ?' p , < t < T. And it 
follows that for < s < t < T, j]' p - J\* > J*' p - J 2 ' p . Let p -> oo, thanks 
to the convergence results, we get that 



9]<9lJl>JlJl-Jl>Jt-J ; 



2 
s ' 
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which implies dJ\ > dJ 2 . From fllOp . we know that 

M«l) * A 1 Jvi dJl 



Y * = f Z i = dK % 



so YJ 1 < Y 4 2 and dK\ > dK 2 , which implies that K\ > K| in view of 
K\ = K 2 = 0. □ 

From this result, we prove the following comparison theorem when the 
coefficient / satisfies monotonicity and general increasing condition in y, and 
quadratic increasing in z. 

Proposition 6.2 Suppose that and f l (s,y,z), i = 1,2 satisfy the condi- 
tion Asssumption 1 and 2, L satisfies Assumption 3. The two triples 
(Y 1 , Z l , K l ), (T 2 , Z 2 , K 2 ) are respectively the solutions of the RBSDE^ 1 , f 1 , L) 
and RBSDE(£ 2 , f 2 , L). If we have V(t, y, z) G [0, T\ x R x R rf , 

f\t,y,z)<f 2 (t, y ,z), e<e, 

then < Y 2 , K\ > K 2 and dK} > dK 2 , for t G [0, T\. 

Proof. First with changement of (Y,Z,K), 

(Y\z\K b ) = (Y -b, Z,K), 

we work with L b < 0. Since this transformation doesn't change the mono- 
tonicity, then in the following, we assume that the barrier L is a negative 
bounded process. As in the proof of theorem I3.1[ for C G R + , let g c : R — > 
[0, 1] continuous which satisfies (j3J. Set y, = g c (x)f l (t, y, z), i = 1, 2, 
which satisfies Assumption 7, with l l (y) = ip l (\2C\). We consider solutions 
(Y*' , Z^ c , K^ c ) of the RBSDE(f ,/f , L 6 ) respectively. Using proposition 
16.11 since 

f?{t, y ,z)<ff(t iyi zi e<e, 

we get for t G [0,T], 

yri,c < Y 2 ' c ,dK}' c > dK?' c . 



Then by the bounded property of Y l , we choose C big enough like in the 
proof of theorem 13.11 which follows immediately 

Y?<Y?,dKl>dK$yte[Q,T\. 

□ 
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Proposition 6.3 Suppose that £ l e L 2 (JF T ) ; f l (s,y,z) satisfy the condition 
Assumption 6, and L l satisfies Assumption 3, i = 1,2. TTie two triples 
(Y 1 , Z X ,K X ), (Y 2 , Z 2 ,K 2 ) are respectively solutions of the RBSDEfe 1 , f\ L) 
and RBSDE((?, f 2 , L). If we have for V(t, y, z) e [0, T] x R x R d , 

e<e, f 1 (t iyi z)<f 2 (t,y,z),L 1 t <L 2 , 

then Yl < Y 2 , forte [0,T]. 

The result comes from the comparison theorem in [8] and the approxima- 
tion in the proof of theorem 15.11 
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